Dimensionality Reduction Basics Quiz

1. What is the primary goal of dimensionality reduction in feature engineering?

Decrease the number of features while preserving information

Increase the model's computational cost

Remove all categorical variables

Ensure all features are normally distributed

Dimensionality reduction aims to simplify datasets by reducing the number of features, making them easier to analyze and visualize. This process retains essential information, helping models perform better and faster by eliminating redundancy and noise, thus enhancing overall efficiency without compromising the quality of insights derived from the data.

Explanation

Dimensionality reduction aims to simplify datasets by reducing the number of features, making them easier to analyze and visualize. This process retains essential information, helping models perform better and faster by eliminating redundancy and noise, thus enhancing overall efficiency without compromising the quality of insights derived from the data.

2. Which dimensionality reduction technique is unsupervised and finds principal components?

Logistic Regression

Principal Component Analysis (PCA)

Random Forest

Linear Discriminant Analysis

Principal Component Analysis (PCA) is an unsupervised dimensionality reduction technique that transforms high-dimensional data into lower dimensions by identifying the directions (principal components) that maximize variance. It does not rely on labeled data, making it effective for exploratory data analysis and feature reduction while preserving essential patterns in the dataset.

Explanation

Principal Component Analysis (PCA) is an unsupervised dimensionality reduction technique that transforms high-dimensional data into lower dimensions by identifying the directions (principal components) that maximize variance. It does not rely on labeled data, making it effective for exploratory data analysis and feature reduction while preserving essential patterns in the dataset.

3. In PCA, what does the first principal component represent?

The direction of maximum variance in the data

The feature with the highest correlation

The most frequently occurring value

The target variable's mean

In Principal Component Analysis (PCA), the first principal component captures the direction in which the data varies the most. It is a linear combination of the original features that maximizes variance, thereby summarizing the most significant patterns in the dataset while reducing dimensionality.

Explanation

In Principal Component Analysis (PCA), the first principal component captures the direction in which the data varies the most. It is a linear combination of the original features that maximizes variance, thereby summarizing the most significant patterns in the dataset while reducing dimensionality.

4. Which method selects a subset of the original features without transformation?

PCA

Feature Selection

T-SNE

Autoencoders

Feature selection is a process that involves selecting a subset of relevant features from the original dataset without altering their values. This method helps to improve model performance and reduce overfitting by retaining only the most informative variables, unlike techniques like PCA or autoencoders that transform the original features into new representations.

Explanation

Feature selection is a process that involves selecting a subset of relevant features from the original dataset without altering their values. This method helps to improve model performance and reduce overfitting by retaining only the most informative variables, unlike techniques like PCA or autoencoders that transform the original features into new representations.

5. What is the 'curse of dimensionality'?

Models perform worse with too many irrelevant features

Features must always be scaled

All features must be highly correlated

Dimensionality reduction is always harmful

The 'curse of dimensionality' refers to the phenomenon where the performance of machine learning models deteriorates as the number of features increases, particularly when many of those features are irrelevant. This can lead to overfitting, increased computational cost, and difficulty in finding patterns, making it crucial to manage feature selection effectively.

Explanation

The 'curse of dimensionality' refers to the phenomenon where the performance of machine learning models deteriorates as the number of features increases, particularly when many of those features are irrelevant. This can lead to overfitting, increased computational cost, and difficulty in finding patterns, making it crucial to manage feature selection effectively.

6. Which feature selection method uses model coefficients to rank feature importance?

Variance Threshold

Mutual Information

Permutation Importance

Coefficient-based Selection

Coefficient-based Selection ranks feature importance by evaluating the coefficients assigned to each feature in a model. Features with larger absolute coefficients are considered more influential in predicting the target variable, allowing for a clear understanding of which features contribute most significantly to the model's performance. This method is particularly effective in linear models.

Explanation

Coefficient-based Selection ranks feature importance by evaluating the coefficients assigned to each feature in a model. Features with larger absolute coefficients are considered more influential in predicting the target variable, allowing for a clear understanding of which features contribute most significantly to the model's performance. This method is particularly effective in linear models.

7. How does t-SNE differ from PCA?

T-SNE is linear; PCA is nonlinear

T-SNE preserves local structure; PCA preserves global variance

T-SNE requires labeled data; PCA does not

T-SNE is faster than PCA on large datasets

t-SNE (t-distributed Stochastic Neighbor Embedding) focuses on maintaining the local relationships between data points, making it effective for visualizing clusters. In contrast, PCA (Principal Component Analysis) emphasizes capturing global variance across the entire dataset, which can overlook smaller, localized patterns. This fundamental difference in focus leads to distinct applications for each technique in data analysis.

Explanation

t-SNE (t-distributed Stochastic Neighbor Embedding) focuses on maintaining the local relationships between data points, making it effective for visualizing clusters. In contrast, PCA (Principal Component Analysis) emphasizes capturing global variance across the entire dataset, which can overlook smaller, localized patterns. This fundamental difference in focus leads to distinct applications for each technique in data analysis.

8. What does explained variance ratio tell you in PCA?

The correlation between features

The proportion of variance captured by each component

The prediction error of the model

The number of outliers in the data

Explained variance ratio in PCA quantifies how much of the total data variability is accounted for by each principal component. It helps determine the effectiveness of the components in representing the original dataset, guiding decisions on how many components to retain for analysis while minimizing information loss.

Explanation

Explained variance ratio in PCA quantifies how much of the total data variability is accounted for by each principal component. It helps determine the effectiveness of the components in representing the original dataset, guiding decisions on how many components to retain for analysis while minimizing information loss.

9. Which technique removes features with low variance across samples?

Variance Threshold

Recursive Feature Elimination

Principal Component Analysis

Feature Scaling

Variance Threshold is a feature selection technique that eliminates features whose variance falls below a specified threshold. This approach helps to reduce the dimensionality of the dataset by removing redundant features that do not contribute significantly to the model's predictive power, thereby enhancing model efficiency and performance.

Explanation

Variance Threshold is a feature selection technique that eliminates features whose variance falls below a specified threshold. This approach helps to reduce the dimensionality of the dataset by removing redundant features that do not contribute significantly to the model's predictive power, thereby enhancing model efficiency and performance.

10. In feature engineering, what does multicollinearity refer to?

High correlation between two or more features

Features with missing values

Categorical features that need encoding

The number of rows in the dataset

Multicollinearity occurs when two or more features in a dataset are highly correlated, meaning they provide redundant information. This can lead to issues in regression models, such as inflated standard errors and unreliable coefficient estimates, making it difficult to determine the individual effect of each feature on the target variable.

Explanation

Multicollinearity occurs when two or more features in a dataset are highly correlated, meaning they provide redundant information. This can lead to issues in regression models, such as inflated standard errors and unreliable coefficient estimates, making it difficult to determine the individual effect of each feature on the target variable.

11. Which method uses tree-based models to determine feature importance?

Standardization

Permutation Importance

Min-Max Scaling

One-Hot Encoding

Permutation Importance assesses the impact of individual features on a model's performance by measuring the change in accuracy when the values of a feature are randomly shuffled. This method leverages tree-based models to evaluate how much each feature contributes to the predictive power, allowing for a clear understanding of feature importance.

Explanation

Permutation Importance assesses the impact of individual features on a model's performance by measuring the change in accuracy when the values of a feature are randomly shuffled. This method leverages tree-based models to evaluate how much each feature contributes to the predictive power, allowing for a clear understanding of feature importance.

12. What is a key advantage of using PCA for dimensionality reduction?

It improves model accuracy guaranteed

It reduces computation and memory while preserving variance

It handles missing values automatically

It eliminates the need for feature scaling

PCA, or Principal Component Analysis, simplifies datasets by transforming them into a lower-dimensional space while retaining the most significant variance. This reduction minimizes computational load and memory usage, making it efficient for analysis without losing essential information, thus enhancing the overall performance of machine learning models.

Explanation

PCA, or Principal Component Analysis, simplifies datasets by transforming them into a lower-dimensional space while retaining the most significant variance. This reduction minimizes computational load and memory usage, making it efficient for analysis without losing essential information, thus enhancing the overall performance of machine learning models.