Machine Learning Model Basics Quiz

Accuracy measures the overall correctness of a model by calculating the ratio of correct predictions to the total number of predictions made. It provides a straightforward indication of a model's performance across all classes, reflecting how often the model is right in its predictions.

A true positive in a confusion matrix indicates instances where the model accurately identifies a positive outcome. This means that the model's prediction aligns with the actual positive cases, reflecting its effectiveness in recognizing the desired condition or event within the dataset.

Precision measures the accuracy of positive predictions made by a model. It answers the question of how many of the predicted positive cases were indeed correct, providing insight into the model's reliability in identifying true positives among all positive predictions. A higher precision indicates fewer false positives.

Recall measures the ability of a model to identify all relevant instances within a dataset. It is calculated as the ratio of true positives (correctly identified positive cases) to the total number of actual positive cases, including both true positives and false negatives. This metric highlights the model's effectiveness in capturing positive outcomes.

Overfitting happens when a model learns the training data too well, capturing noise and outliers instead of general patterns. As a result, it excels on the training set but fails to generalize to new, unseen data, leading to poor performance on test data. This imbalance indicates that the model lacks robustness.

Splitting data into training and test sets allows for the assessment of a model's effectiveness on new, unseen data. This process helps ensure that the model generalizes well and is not merely memorizing the training data, thus providing a clearer indication of its predictive capabilities in real-world scenarios.

The F1-Score combines precision and recall to provide a single metric that balances both the accuracy of positive predictions (precision) and the ability to identify all relevant instances (recall). By using the harmonic mean, it ensures that both metrics are considered equally, emphasizing the importance of achieving a good balance between them.

F1-Score is ideal for imbalanced datasets because it balances precision and recall, providing a better measure of a model's performance on the minority class. Unlike accuracy, which can be misleading when one class is rare, F1-Score focuses on the model's ability to correctly identify both classes, making it more informative in such scenarios.

Cross-validation is a technique used to assess how a model will generalize to an independent dataset. By dividing the data into multiple subsets, it trains the model on some subsets while testing it on others. This process provides a more reliable estimate of model performance, reducing the risk of overfitting to a single data split.

ROC-AUC measures the trade-off between true positive and false positive rates across different thresholds. It evaluates how well a model distinguishes between classes, with a higher AUC indicating better performance. This metric is particularly useful for assessing models in binary classification tasks, providing insight into their ability to balance sensitivity and specificity.

A model predicting every case as positive identifies all actual positives, resulting in high recall, which measures the ability to find all relevant instances. However, since it also incorrectly identifies many negatives as positives, precision suffers, indicating a high number of false positives relative to true positives.

Regularization techniques, such as L1 (Lasso) and L2 (Ridge), introduce a penalty for larger coefficients in a model. This discourages overly complex models that fit the training data too closely, thus reducing the risk of overfitting and improving the model's generalization to unseen data.