Apache Spark Machine Learning IBM AI Engineering

1. In machine learning, what does the training set primarily do?

Evaluates model performance

Fits model parameters

Tests final accuracy

Defines feature types

2. Which Spark MLlib algorithm is used for regression tasks?

K-means

Linear regression

Apriori

Hierarchical clustering

3. What is the purpose of feature engineering in machine learning?

Increase model complexity unnecessarily

Create meaningful representations of raw data

Reduce all features to one dimension

Eliminate the need for training

4. In Spark SQL, what does the join operation combine?

Multiple DataFrames based on common keys

Rows within a single DataFrame

Column names only

Metadata and schemas

5. Which Spark operation is lazy and not executed immediately?

Collect()

Map()

Show()

Count()

6. What is the primary function of hyperparameter tuning in ML models?

Increase data size

Find optimal model configuration for best performance

Reduce computational cost only

Eliminate missing values

7. In MLlib pipelines, what is a Transformer?

A neural network layer

A component that converts data into features or predictions

A database connection handler

A file format converter

8. What does a Spark RDD transformation create?

A new RDD from an existing one

A permanent database table

A broadcast variable

An immediate result

9. Which evaluation metric is typically used for binary classification problems?

Silhouette score

Accuracy and AUC-ROC

Inertia

Entropy only

10. What is the role of a Spark partitioner in distributed processing?

Compresses data files

Distributes data across cluster nodes

Encrypts sensitive information

Removes duplicate records

11. What is the primary advantage of using Apache Spark over Hadoop MapReduce for machine learning?

In-memory processing and faster iteration

Lower cost of hardware

Better support for SQL queries

Smaller data file sizes

12. What is a Spark DataFrame schema?

The physical storage format of data

The structure defining column names and data types

A backup copy of the data

The execution plan for queries

13. Which MLlib algorithm is used for unsupervised learning to group similar data points?

Support vector machines

K-means clustering

Random forest

Naive Bayes

14. In Spark MLlib, what does cross-validation help prevent?

Data loss

Overfitting

Memory errors

Network latency

15. Which Spark action returns all RDD elements to the driver program?

Collect()

Take()

First()

Count()

16. What is the purpose of feature scaling in machine learning pipelines?

Reduce model training time

Normalize feature values to a similar range

Increase dataset size

Eliminate outliers automatically

17. In MLlib, which algorithm is commonly used for classification tasks?

K-means clustering

Logistic regression

Principal component analysis

Apriori

18. What transformation operation in Spark applies a function to each element and returns a new RDD?

Filter()

Map()

Collect()

Reduce()

19. Which Spark data structure is optimized for structured data and provides SQL query capabilities?

RDD

DataFrame

Broadcast variable

Accumulator

20. In Spark MLlib, what does an RDD (Resilient Distributed Dataset) represent?

An immutable, distributed collection of objects

A relational database table

A single-machine data structure

A temporary cache for SQL queries

IBM AI Engineering Apache Spark Machine Learning Quiz

1. In machine learning, what does the training set primarily do?

2.

What first name or nickname would you like us to use?

2. Which Spark MLlib algorithm is used for regression tasks?

3. What is the purpose of feature engineering in machine learning?

4. In Spark SQL, what does the join operation combine?

5. Which Spark operation is lazy and not executed immediately?

6. What is the primary function of hyperparameter tuning in ML models?

7. In MLlib pipelines, what is a Transformer?

8. What does a Spark RDD transformation create?

9. Which evaluation metric is typically used for binary classification problems?

10. What is the role of a Spark partitioner in distributed processing?

11. What is the primary advantage of using Apache Spark over Hadoop MapReduce for machine learning?

12. What is a Spark DataFrame schema?

13. Which MLlib algorithm is used for unsupervised learning to group similar data points?

14. In Spark MLlib, what does cross-validation help prevent?

15. Which Spark action returns all RDD elements to the driver program?

16. What is the purpose of feature scaling in machine learning pipelines?

17. In MLlib, which algorithm is commonly used for classification tasks?

18. What transformation operation in Spark applies a function to each element and returns a new RDD?

19. Which Spark data structure is optimized for structured data and provides SQL query capabilities?

20. In Spark MLlib, what does an RDD (Resilient Distributed Dataset) represent?