Databricks Data Engineer ETL Pipeline Quiz

1. In Spark, what is a DataFrame?

A Python-only data structure

Distributed immutable collection with named columns

A relational database table

A single-machine data structure

2. In Databricks notebooks, what does %sql magic command do?

Enables Python code execution

Runs SQL queries directly in the notebook

Compiles Scala code

Manages cluster configurations

3. What is the main difference between batch and streaming ETL in Databricks?

Batch processes historical data; streaming processes continuous data

Streaming is always faster

Batch is more reliable

Streaming cannot handle transformations

4. Which operation in Spark SQL filters rows based on a condition?

SELECT

WHERE

GROUP BY

ORDER BY

5. In Databricks, what is Unity Catalog used for?

Managing compute clusters only

Centralized governance and access control for data

Writing SQL queries exclusively

Scheduling batch jobs

6. What is the purpose of the VACUUM command in Delta Lake?

Removes outdated data files and reclaims storage

Validates table schema

Encrypts sensitive data

Restores deleted rows

7. Which Spark transformation is lazy and not executed immediately?

Collect()

Show()

Map()

Count()

8. In ETL pipelines, what is data lineage tracking?

Recording the origin and transformations of data

Deleting historical data versions

Encrypting sensitive columns

Measuring query execution time

9. What does the Z-ORDER BY optimization do in Delta Lake?

Sorts data alphabetically

Clusters related data together for faster queries

Removes duplicate records

Converts data types automatically

10. Which Databricks workspace feature helps organize ETL notebooks and jobs?

Folders and permissions

Encryption keys only

Cluster pools exclusively

Job parameter validation

11. What is the primary advantage of Delta Lake over standard data lakes?

ACID transactions and schema enforcement

Lower storage costs only

Faster read operations exclusively

Eliminates the need for data validation

12. How does Delta Lake handle schema evolution in ETL pipelines?

Rejects new columns automatically

Supports adding new columns while maintaining compatibility

Requires manual schema updates

Deletes all existing data

13. What is the role of a cluster in Databricks ETL execution?

Provides compute resources for running Spark jobs

Stores all pipeline metadata

Manages user authentication only

Compresses data automatically

14. Which data format does Databricks recommend for production ETL pipelines?

CSV

JSON

Delta

XML

15. In Databricks, what is the main benefit of using Structured Streaming?

Processes real-time data with fault tolerance

Eliminates the need for batch processing

Reduces storage requirements by 50%

Automatically handles all data quality issues

16. What does the MERGE statement in Delta Lake accomplish?

Combines multiple tables into one

Performs upsert operations (insert/update/delete)

Joins data from two sources without updating

Splits large tables into partitions

17. Which Databricks feature enables scheduling and monitoring of ETL jobs?

Databricks SQL Editor

Databricks Jobs and Workflows

Delta Lake Streaming

Apache Airflow integration only

18. What is the purpose of the OPTIMIZE command in Delta Lake?

Compacts small files and improves query performance

Deletes all historical versions of data

Converts Delta tables to Parquet format

Encrypts table metadata

19. In Spark SQL, what does the PARTITION BY clause do in an ETL pipeline?

Divides data into logical groups for parallel processing

Encrypts sensitive data columns

Removes duplicate rows automatically

Compresses data for storage

20. Which command creates a managed Delta table in Databricks?

CREATE EXTERNAL TABLE

CREATE TABLE USING DELTA

CREATE DELTA MANAGED

REGISTER DELTA TABLE

Databricks Data Engineer ETL Pipeline Development Quiz