PySpark Notebook Basics for Fabric Analytics

1. What does 'schema' define in a PySpark DataFrame?

The structure of columns and data types

The number of rows

The storage location

The cluster configuration

2. In Fabric notebooks, 'spark' is a pre-initialized ____.

SparkSession

DataFrame

Partition

Cluster

3. Which method adds a new column to a PySpark DataFrame?

WithColumn()

AddColumn()

InsertColumn()

CreateColumn()

4. In PySpark, what does 'broadcast()' optimize?

Distributing small DataFrames to all worker nodes efficiently

Merging large tables

Streaming data processing

Compressing files

5. True or False: The 'collect()' action should be used on large DataFrames without caution.

True

False

6. Which method removes duplicate rows from a DataFrame?

Distinct()

Unique()

Deduplicate()

Remove_duplicates()

7. What does 'wide transformation' mean in Spark?

Requires data shuffling across partitions

Operates only on single partitions

Increases DataFrame width only

Reduces memory usage

8. In Microsoft Fabric notebooks, which language is primarily used alongside PySpark?

Python

Java

Scala

R

9. True or False: Spark transformations return new DataFrames without modifying the original.

True

False

10. Which method displays the first N rows of a DataFrame?

Show()

Head()

Display()

Print()

11. In Microsoft Fabric, what is the primary purpose of a PySpark notebook?

Execute distributed data processing and transformations using Python

Create static reports only

Manage user permissions

Store raw data files

12. In PySpark, which operation joins two DataFrames?

Join()

Merge()

Combine()

Union()

13. What is the purpose of the 'groupBy()' method in PySpark?

Aggregate rows by specified columns

Sort data in ascending order

Remove duplicate rows

Combine two DataFrames

14. Which action returns all DataFrame rows to the driver node?

Collect()

Gather()

Fetch()

Retrieve()

15. In Spark, what does 'partitioning' refer to?

Dividing data into chunks distributed across cluster nodes

Splitting a notebook into sections

Separating data types in memory

Creating backups of data

16. Which PySpark transformation filters rows based on a condition?

Filter()

Select()

Map()

Reduce()

17. What is the correct syntax to select multiple columns from a PySpark DataFrame?

Df.select('col1', 'col2')

Df.get('col1', 'col2')

Df.pick('col1', 'col2')

Df.choose('col1', 'col2')

18. In PySpark, which method creates a DataFrame from a CSV file in Fabric?

Spark.read.csv()

Spark.load.csv()

Spark.import.csv()

Spark.parse.csv()

19. What does 'lazy evaluation' mean in PySpark?

Transformations execute only when an action is called

Data is evaluated immediately upon creation

Notebooks run without processing data

Actions delay execution indefinitely

20. Which Spark object represents an immutable distributed collection of data?

RDD (Resilient Distributed Dataset)